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(57) Abstract 

The present invention 
teaches a variety of methods 
and systems for providing 
computei/human interfaces. 
According to one method, the 
user interfaces with an elcctnxiic 
device such as a computer 
system by engaging a sensor 
with desired regions of an 
encoded physical medium. "Hie 
encoded physical medium is 
preferably chosen to provide 
intuitive meaning to the user, and 
is thus an improved metaphor 

system. Suitable cjcampiea of 
the encoded physical medium 
include a data-linked book, 
magazme. globe, or articles 
of clothing. Some or all of 
the selected regions have had 
certain information encoded 
therehi. Infomutlon suitable for 
interfacing and controlling the 
computer system. When the user 
engages die sensor with a region 

' ^^'^"c^^;^^'^:;-^;^^:^^^-^^ "--P-^i « appropriate ^ ckcn. Fo, cxa„.p.c. *c 

ilhiJtration, that<liw ihl ^ TXl^ ^^J^,^ "T ^ '""x*"" l*y»ical medunn may have text and/or gn^hic 

storing the certain encodedTta i, a ^1*, ^^^^J^'^.^T^'^^'^r^^'^ " tafomation. One sch^for 

Weue (RGB). ITie content is cnc^^^i^l^f ^L JX ^ embodiment, the spectral encoding «*eme uses red. green, 
color is insen^l into the ho. ^^TtTlSTh'at'^co'IlLn.re'n^^^Th^^^ » 




FOR THE PURPOSES OF INFORMATION ONLY 
Codes used to identify States party to the PCT on the front pages of pamphlets publishing intematiocul applications under the PCX. 



AL 


AKMflU 


ES 


Spahi 


LS 


Lesotho 


SI 


Slovenia 


AM 


AnneaU 


FI 


Fmlind 


LT 


IJdaianU 


SK 


SlovakU 


AT 


Antrui 


FR 




LU 


Laxembo«x 


SN 


Senegal 


AU 


Aasonlia 


CA 


Ocbon 


LV 


LatvU 


SZ 


Swazlbad 


A£ 


AzcHnija 


GB 


United fOc^docn 


MC 


Monaco 


TO 


Otad 


BA 


Bo«au utd Henctoviu 


GE 


Oeorcta 


MO 


RepuUie of Moldova 


TC 


TOjjo 


BB 


Btrt>Mlo« 


GH 


Ohau 


MG 


Madatascar 


TJ 


T^^usun 


BE 




GN 


Oaraea. 


MK 


The fanner Yofoslav 


TM 


TMuaeaisUn 


BP 




GR 


Greece 




Repobltc of Macedonia 


TR 


Twfcej- 


BO 


BttlfvU 


IIU 


nwxgfuy 


ML 


Mali 


TT. 


TVinidad and 'Hobafo 


BJ 


Bcntn 
Bnzfl 


re 




MN 


Moncolia 


UA 


Ulnine 


BR 


IL 


brtel 


MR 


MauriUAla 


UG 


Uianda 


BY 


Belafos 


IS 


Iceland 


MW 


MaUwi 


US 


United SUtes of Asnc 


CA 


Canada 


IT 


baly 


MX 


Mcaloo 


UZ 


Usbcfcntaa 


CF 


CcntnJ Afncu Repct>lic 


JP 


Itptn 


NE 


Nlser 


VN 


Viet Nam 


CG 


Conso 


KE 


Kenya 


NL 


Neiherlaiida 


YU 


Yvjoslavia 


CH 


SwiuerUnd 


KG 


Kyts/zuan 


NO 


Norvragr 


zw 


Zimbabwe 


CI 


Cfite dlvoifc 


KP 


DenMcmk People's 


NZ 


New Zealand 






CM 


Cud croon 




Republic of Korea 


PL 


Poland 






ON 


Chtfu 


KK 


(tepubitc o( Korea 


PT 


Poctn^al 






cu 


Oib* 


KZ 


Kntkican 


RO 


Romania 






cz 


Cxixh Republic 


LC 


Saim Locia 


RU 


RuEuan Fcdenticm 






OE 


Gennuiy 


U 


Uechtenstein 


SD 


Sudan 






DK 


Denmark 


LK 


Sri Lanka 


SB 


Sweden 






EE 


EcoaU 


Ul 


Liberia 


SC 


Singipore 







wo 99/18487 



PCTAJS98/20661 



METHODS AND SYSTEMS FOR PROVIDING HUMAN/CO MPUTER 

INTERFACES 



Rackground of the Invention 

This invention relates generally to computer interfaces. More 
specifically, the present invention discloses a variety of computer interfaces 
using encoded physical mediums wherein an encoded region may include a 
marker indicating that information is encoded therein. The present invention 
also teaches data-linked physical mediums that provide users intuitive access to 
multimedia information that may be stored remotely. 

People are constantly interacting with computerized systems, from the 
trivial (c.g^., the computerized toaster or the remote control television) to the 
exceedingly complex (e.g^., telccommuiiicaiions systems and the internet). An 
advantage of computerization is that such systems provide flexibility and power 
to their users. However, the price that must be paid for this power and 
flexibility is, typically, an increase in the difficulty of the human/machine 
interface. 

A fundamental reason for this problem is that computers operate on 
principles based on the abstract concepts of mathematics and logic, while 
hiunans tend to think in a more spatial manner. Often people are more 
comfortable with physical, three-dimensional objects than they are with the 
abstractions of the computer world In short, the power and flexibility provided 
by the computer and related electronic technology are inherently limited by the 
ability of the human user to control these devices. Since people do not think 
like computers, metaphors are adopted to permit people to effectively 
commimicate with computers. In general, better metaphors permit more 
efficient and iiiedium indep end e nt communi<:atioHS between people and 
computers. The better metaphor will provide the user a natural and intuitive 
interface with the computer without sacrificing the computer's potential. 

There are, of course, a number of computer interfaces which allow users, 
with varying degrees of comfort and ease, to interact with computers. For 
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system. However, these computer interfaces T ' ^^"'P"'^^ 

- . .e,.re a su.tan., Jestmer^^Ln^ t^ r^^^^"' 

graphic'Tt"^ .^er^r (GlJ) "ST'S^- '"^^^^^^ the 

objects are displayed on a computer screen P^^^'^^I 
look like a page of a document, a cSecZ file 'IT''' ' ^"'^^^ ^^^^ 
an icon of a trash can may be uZ f^- « fo^er. and 

other word, GUIs use '^l^TJ^rtr^^ of documents and files. I„ 
Object familiar to uset.. ^rmSes oJL eSe'r """"'^^ ^ P^^^^' 
were p.oneered at such places as X^ox P^c of P ^ l?^'" ^^'^ 
Apple Computer. Inc. of Cupertino r«.^^^ ^^"^""^a and 

commonly used with UNl^ ^« ^UI is also often 

standard i„ the PC/MS-DOS worfd wirr^'w""^ '"'"'^'^ ^ 
provided., Microsoft Co^oration^f Rrmond. °^'^^^^ 

present ^^^aces. they nonetheless 

words, an icon can only represent a """^^^ "^-^P^or. In other 

Object. It would be ideaUme "Iputr^^^^^^^ " "^^'^^ Physical 

medium which could convey a f^C '"1 ""^ '"^'"''^^ ^ P^^sical 
ta^k at hand. While progress h^^iln T"^' ^''^''P' ^^'^^^ the 
-any roadblocks yet^emai„. ^'re^^m^^^^^^^ 

application one has selected a Physical ; ^''"^^ ^ 8*ven 

the infonnation necessary to 3™ t^^" " 

encoded within the ph^icS ^^'.^^'"P^ter interface must still be 

developed for linking'^urLtT^^^^ --t be 

available^omremotesourcesus^^^^^^^ 

^tEDlA," filed My I 1994 ^2;/ ^^^ESSING INTERACTIVE 

encoding infonnation wiiiin a Dhv,rr ^ °'"' '^'^tt^^^taiy mechanism for 

printed publication sucT^ " bSe^^ '''''°''' "^^^"^^^ the use of a 

a data button, and remote contr'l^^^^^^^^^ '° ^ ^^^^^^'^ -^^^^ 

ontrol circuitiy. TTie button is physically attached to 
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Summary of the Invpnfinn 

The present invention improves the human/computer interface by 
providing a method for interfacing via an encoded physical medium having a 
region wherein information has been encoded. The interface method includes 
measuring information present in a first region of the encoded physical medium 
and then determining whether the measured information contains a marker 
indicating that certain information has been encoded therein. According to one 
embodiment, the marker is capable of generating light within a particular range 
of electromagnetic wavelengths, either by reflection or through luminescence. 
When the marker is reflective, the sensor typically includes a light emitting 
element and a sensing element. However, when the marker and the encoded 
region are luminescent, the sensor need only include a sensing element. 

In related embodiments of the present invention, the information may be 
encoded according to a spectral encoding scheme, a bar code scheme, or a 
combination thereof. The marker may be infrared ink applied over the certain 
encoded information, regardless of how the certain information is encoded. 

The present invention also teaches that when it is determined that the 
marker is present in the first region, the certain encoded information is 
translated into certain decoded information including a function to be performed 
by the computer system. The fimction to be performed by the computer system 
may include, among other things, providing visual, audio, and/or tactile 
feedback. The certain decoded information could also include a uniform 
resource locator (URL) and the function may involve the computer system 
accessing and/or displaying an Internet web page to which the URL directs. 



The present invention further teaches maintaining a database tracking the 
results of the user engaging the sensor with a plurality of regions, including the 
determination of null meaning region, i.e., regions that do not contain a marker. 
The database could then be used later to determine whether a specific condition 
(such as collection of a fixed number of clues or data points) has been satisfied. 
30 In turn, a specified action could be performed by the sensor of the computer 
system. 
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U«t is .0 be encoded iJ^^i^TtJ^ '^^''"'^ '««™g comem 

encoding fte oonte. atc^j^^ Tr T '""^^ "^'^•"^ "■"'ta™. 
application onto B,. cn'SaXL^rSl'"''^."^ 
content together will, a maAer tamToLr^- """"^ 
«pre«„tadon of the e„c^^"hyl, "S""^' "'f ^>"» = 

content is encoded within the cS^^^^L T' " "^""^ U>e 

»jbse,nc„.,, engag::.t.tr& *rs.r ^^^^^^ 

the representation is creat<»H fi,» "^'^ ^^'stence of the content. Once 

encodedphysica.;iir^;,tgZ::;,rt"' "^"^ 
uin may oe generated from the representation. 

number du. .^resenB tS^l^l'td ^^="""6 = "'""y 

-presents d,ecomen,accorLgX"L^"^? ""^ """"^ 
embodiments, die maritcr reotl^T f ! ^ In sonK„:lated 

pa.icu.ar ra^gc of^^^^oZ^c" Jl," Of '^""L '^'^ ^ 
invention also teaches diat le« and Pf-^"™' 
rept^ntation of die JZJ^LTf^ T"^^'" ™^ """'l^ "il^n d,e 
P-y^ica, n.rbrcrS1ir:r^ht''r?'.*° ~ 

-P.^udo„usingacomp„tersystem„roZ;s;chL " 

^<^'^ZV^rr ' computer 

n>«Uum. The ^c^dTnli ^ =™=oded physical 

a sensor operable for measuri^.^^^^ computer mterface includes 
medium. aTd a IZZ^l^'^'" ""^'^ 
-vhedier infotmation m^ """^"e 
cemin encodedinfott^ST^p^ 

•mbodimcnt, Am computer i^^^^- , '^ '^'^ mfbimation. In a related 
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computer interface also has a transmitter device operable to transmit the certain 
decoded information to the computer system. 

In still another related embodiment, the marker is operable to generate 
light from within a particular range of electromagnetic wavelengths. In this 
5 embodiment, the sensor has a sensing element responsive to the particular range 
of electromagnetic wavelengths. By generate light, it is meant that the marker 
can either reflect and/or emit light. 

In some embodiments, the sensor has an opaque shroud covering the 
sensing element to protect it from ambient light. In other embodiments, the 
10 sensor includes filter circuitry to eliminate noise due to the ambient light. 

One other separate embodiment of the present invention teaches an 
encoded physical medium. The encoded physical medium is suitable for use in 
interfacing a user and a computer system and has a region wherein certain 
mformation is encoded. The certain encoded information includes a marker 
indicating that the certain encoded information is encoded in the first region. 
The certain encoded information includes data suitable for interpreting into 
computer readable data. The encoded physical medium may take on a variety of 
forms such as an article of apparel, packaging material, a book or magazine, and 
a globe. The certain encoded information may be encoded according to a bar 
code or spectral encoding scheme, the spectral encoding scheme including 
encoding colors red, green, and blue, and possibly some colors chosen from the 
infixed color range. 

Still another separate embodiment of the present invention teaches an 
electronic data linked physical medium suitable for linking a physical medium 
with video and audio data stored on multimedia networked computers. 
Typically, the data linked medium includes a physical medium having at least 
one hot spot encoded with linking data enabling the data linked physical 
medmm to access remote data, a sensor operable to measure and decode the 
linking data, and a transmitter operable to transmit the linking data to a remote 
computer system. The remote computer system is responsive to the linking data 
to retrieve the remote data and present it to a user of the data linked physical 
medium. Thus the user of the data linked physical medium is provided a 
mechanism for linking to and accessing remote data. 
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The present invention therefore provides a more intuitive and richer 
metaphor for the interaction between humans and computerized systems. These 
and other advantages of the present invention will become apparent upon 
reading the following detailed descriptions and studying the various figures of 
the drawings. 
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Bri^f Description of th£.Drawings 



present invention. ^ " accordance with one aspect of the 

present invention. accordance with another aspect of the 

one eI'S.:„;;?^™:«'— ' wiU, 

— r:.;::et=fjrpt:-- - ^^^^ — 

M Jf^I:;: '"--'O" of a ho. spo. .He 

invention. ™ another aspect of the present 

^bodtaT j"4i^^^ of a fc, ^.a lu^ 
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Detailed Description of the Preferred Embodiments 

With reference to Figure 1 , a computer interface 1 0 in accordance with 
one embodiment of the present invention will now be described. The interface 
10 includes a sensor 12 and an encoded physical medium 30. As will be 
apparent, the computer interface 10 provides an intuitive mechanism for a user 
to interface with and control an electronic device such as a computer system 40 
(also illustrated in Figure 1). 

The encoded physical medium 30 has at least one region 32 wherein 
information has been encoded. The content of the region 32 may be encoded 
according to a well known content encoding scheme such as a bar code scheme. 
The present invention also teaches a variety of new encoding schemes. For 
example, a content encoding scheme contemplated by the present invention is a 
bar code printed using invisible, e.g. infrared (IR), inks. Such a bar code would 
be apparent to the sensor but invisible to the user. Alternatively, the content of 
the region 32 may be encoded according to a spectral encoding scheme. Several 
specific examples of suitable spectral encoding schemes are described below 
with reference to Figures 4-8. The encoded physical medium 30 may also 
include a document identification (ED) hotspot 33, similar to the region 32. The 
content encoded within the document ID hotspot 33 will, however, be special in 
that it provides an indication of the identity of the particular encoded physical 
medium 30. 

In certain embodiments, encoded regions such as the region 32 fiirther 
include a marker 34 indicating that certain encoded information is present in the 
region 32. By way of example, in one spectral encoding scheme, the desired 
content for the region 32 is represented according to different visible and 
infrared inks (reflective and/or photolmninesccnt) applied to the region 32. The 
marker 34 is represented by yet another infixed ink similarly applied to the 
region 34. In this example, the user would not see the marker 34. but may or 
may not see a visual representation of the content encoded in the region 32. 
Throughout the specification, markers, hotspots, regions, inks, etc., arc often 
described as being able to generate Ught. Being able to "generate light" is 
defined herein as including at least one of the ability to reflect or emit light. 
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The encoded physical medium 30 may take any suitable fomi. By way of 
example, the encoded physical medium 30 may be a page from a physical book 
or magazine, an article of clothing such as a T-shirt, a globe, consumer product 
packaging, etc. Such encoded physical mediums may have been marked and 
encoded with data for the specific purpose of providing the interface of the 
present invention. Alternatively, the encoded physical medium 30 may simply 
be items selected from a user's siuroundings, the encoded information arising 
randomly or unintentionally (at least with relation to the user's application). In 
another embodiment, the encoded content arises randomly or unintentionally, 
but a marker 34 is applied intentionally. Some representative examples of 
encoded physical mediums are described below with reference to Figures 10-1 1. 

The sensor 12 includes a sensing element 13, a decoder 14, a transceiver 
16, an on/off switch 18, and memory 20, The sensing element 13 is arranged to 
measure information present on the encoded physical medium 30. When the 
utilized encoding scheme implements a marker 34, the decoder 14 is arranged to 
determine whether the marker 34 is present in measured mformation, and when 
the marker 34 is sensed, to decode the measured information. The transceiver 
16 is operable to transmit data such as decoded information to the computer 
system 40. Depending upon the embodiment, the transceiver 16 may establish 
either a uni-directional or bi-directional communications link 22 between the 
mterface 10 and the computer system 40. The communications link 22 is 
preferably a wireless communications link such as one based upon infrared (IR), 
radio-frequency (RF), or ultrasonic commxmications technology. However, the 
communications link 22 may take the form of a wired communications link 
such as a serial RS-232 or RS-485 data cable, or a parallel data cable. 

In some embodiments, the sensor 12 operates by measuring information 
selected from the encoded physical medium 30 by the user, decoding as 
necessary, and then transmitting the decoded information to the computer 
system 40 for further processing. In other embodiments, the sensor 12 includes 
further hardware for processing the nieasured information locally. For example, 
the sensor 12 may include a microcontroller such as a PIC microcontroller 
programmed to process the measured information. The decoder 14 may be part 
of the microcontroller, or may be separate circuitry. In some embodiments, the 
sensor 12 maintains in the memory 20 a database related to the measured 
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information. The information stored in the database may be used locally at the 
sensor and/or saved for subsequent transmission. 

The computer system 40 appears in Figure 1 as a personal desktop 
computer. However, it is contemplated that the interface 10 is suitable for use 

5 with a wide scope of electronic devices. The wide scope of suitable computer 
systems encompasses all types of personal computers, interactive TV systems, 
set-top boxes, web interfaces, haptic interfaces, streaming music and video 
SQurces, and many others. One particular example is a WebTV "net-top box," 
Further, although the interface 10 is intended to be representative of and thus 

10 generic to a broad range of interfaces contemplated by the present invention, it 
will be appreciated that computer interfaces of the present invention may take 
many forms which go beyond the example interface 10 of Figure 1. 

With reference to Figure 2, a method 100 for providing an interface 
between a user and a computer system in accordance with one aspect of the 

15 present invention will now be described. An initial step 102 provides the user 
with an encoded physical medium 30 and a sensor 12 suitable for sensing 
information present within the encoded physical medium 30. As described 
above, the encoded physical medivmi 30 may take any suitable form. One 
suitable method for perfomiing the step 102 is described in more detail below 

20 with reference to Figure 3. 

In a next step 104, the user explores the encoded physical medium 30 and 
selects a desired portion 32 of the encoded physical medium 30. The user may 
be drawn into the desired portion 32 through text, coloring, or graphics 
illustrated on the desired portion 32. The text, coloring or graphics illustrated 

25 on the desired portion 32 may represent the encoded information, may be in 
addition to the encoded information, or may be some suitable combination of 
illustration and encoded information. Instead of being drawn in, perhaps in the 
case of a game or work task, the user may be selecting the desired portion 32 
based upon some predefined criteria. For example, the user may be searching 

30 for a clue to a puzzle game, or simply doing inventory and selecting a bar code 
foxmd on a product during this process. In any event, once the user has selected 
the desired portion 32 in step 104, in a step 106 the user engages the sensor 12 
with the desired portion 32 of the encoded physical medium 30. The sensor 
engagement of step 106 will typically involve the user setting the sensor 12 to 
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10 



20 



25 



an ON state that indicates that the sensor 12 should be in operation. In the case 
of the interface 10 of Figure 1, the step 106 would involve operation of the 
on/off switch 18. Depending upon the particular sensor and the application, 
sensor engagement may require the user to bring the sensor 12 into close 
proximity to or in contact with the desired region 32. 

In a next step 108, the sensor 12 measures information present within the 
desired region 32 of the encoded physical medium 30. Information is defined 
herein as any data that the sensor 12 is capable of measuring. Thus, the 
information measured by the sensor 12 is not limited to information that has 
been purposefully encoded into the desired region 32 of the encoded. physical 
medium 30. 



A step 110 then determines whether the measured information has null 
meaning. When step 110 determines that the measured information has null 
meaning, it is simply determinmg that the measured information has not been 
15 marked, for the present purposes, as containing encoded infbtraaiion. However, 
as will be appreciated, a determination of null meaning may be consequential. 
Accordingly, when step 1 10 determines that the measured information has null 
meanmg, a step 1 12 performs any action indicated by such a determination. For 
example, the sensor 12 may be equipped with a light that blinks or a buzzer that 
sounds when the measured information has null meaning. As another example, 
the sensor 12 may have memory 20 that is utilized to keep track of the meaning 
of the different measured regions, including each null determination. 
Alternatively, the null information may be transmitted to the computer system 
40. In any event, once step 1 12 is complete, the control of the method 100 is 
passed back to step 104 allowing the user to explore fiirther and select another 
region 32 for sensing. 

When it is determined in step 110 that the information measured in step 
108 does not have null meaning (e.g., the measured information has been 
mariced as containing encoded infbiroa&on), oMfiari is isis^ to a st^ 114 
30 wherein the sensor 12 interprets the measured information. Depending upon the 
specific application, step 1 14 may involve just decoding of the information from 
the particular encoding scheme into a data format suitable for transmission by 
the transceiver 16. However, in some embodiments significant processing of 
the measured information will occur locally at the sensor 12. Thus in a next 

- 12 - 
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Step 116, the sensor 12 performs an operalion that is a function of both the 
information interpreted in the step 1 14 and the context in which the information 
was measured. Note that context depends upon the particular application and 
may include the nature of previously interpreted information, the timing of the 
user's engagement of the sensor 12, information received at the sensor from the 
computer system 40, etc. 

For example, with each new engagement of the sensor 12, the sensor 12 
may store the interpreted information in a database in the memory 20 and then 
evaluate the database or a portion of it to determine whether a predefined 
condition has been satisfied. A predefined condition could be the user gathering 
a set number of clues or data points, at which point the sensor transmits all or 
some of the stored information to the computer system 40. In one specific 
example, the user may be perusing an encoded catalog magazine 30 with a 
hand-held wand sensor 12. As the user engages the wand sensor 12 with 
regions of the catalog 30 representing desired products, these regions are sensed 
and"th c-iiifonuati on therein interpreted by the wand sensor 12. When finished, 
the user may select an order region 32 indicating to the sensor that the user is 
ready to order and purchase the selected items. At this point, the 
communication link 22 could be established with the computer system 40, 
which may be local or remote, and the user*s order information could be 
transmitted to the computer system 40 which in turn could process the order or 
further transmit the order as necessary. 

In other embodiments, the indicated action of step 116 includes the 
computer system 40 and/or the sensor 12 responding to the measured 
information by providing feedback. The feedback could take any suitable form 
such as audio, visual or tactile feedback. In any event, once the indicated action 
has been performed in the step 116, the control of the method 100 is passed 
back to step 104 allowing the user to fiirther explore the encoded physical 
medium 30 and select other regions for sensing. 

As noted above with reference to Figure I, certain encoded physical 
mediums 30 include a document ID hot spot 33. In these embodiments, when a 
user first begins exploring the encoded physical medium 30 as described in step 
104, the initial step 106 ought to be the engagement of the sensor 12 with the 
document ID hot spot 33. Then in steps 1 14-1 16, the sensor 12 could store the 
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Partcular region wherein contents * ^P"'" *fl»ed Ja 

faphics. TT,e encoded com^T of LZ «>**^ 
fonns, dependen, upon such .his as^he !^ ^ » ^ri""/ of 

Oes-gner. For example, tf,e Sta, ^"^ «™k of the 

comen, visually blends togeto ™^*^;'™ ""J *= 
^«.gner Al,en,a«vely, .he encod^g scTLfrL? ^^"'""^ 
bemg usually dis.i„cUve or col e,r S^ conten, 
•■epresena,io„ofu,epi,y,i^^, '3!''=''' ^"'^ tbe viewer, ^e 

or custom made desi^ so^a^^sTm ..""l!! """" ^" °«'-*-'Sf 

A^^^emanvely, the designer may be pravidS""^^^ content of the hot spots, 
hotspol ,s tt,e infonnation to be ^^^Z '^'^'^ 
~„s. a uniform ,esom« U,^C^^:,^Zr7L 

automated such that i.^Z^^^^'^'^' *« «^ wfll b. 
■"••■ate the encoding pr«:ess Sn?* '^ and 
or ™me other suitably pro^S tLT T ' 'y«". 

process will also intriXceT^jtto 1 ^ 
-a.n tafotmation is encoded rS:rho"Tpot ^1??' 

»pois. unce the content is encoded, in 
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lh« encoding scheme is a sdLL7™Th ^.^P'"^ ''=='8" software and 
nece..a:,inLcanpJUr.::drdX"LrL^L^r^ ""•'^^ 

encoding fi.^ 
wita a ho. spot 220. Tfe sensor "decodes" tee vd^l 
mlensities of three difleren, encodmg coloiTci r, 1, / ">«="™g <he 
hot spot 220. CI C2 and n m,„ f . ' ^ " "i"™ 

blue (RGB). Al,™^vdy a^2 1'^" ' 1°"°^°"" »^ 
visible light range (e ^ ^ ^"^ °<'<^i<= <he 

invisible colo^ ^ ' '^mbmation of visible and 

C3. Take the ilZJ^, cl^2 SrT"'^'"'' "^"^ C>. C2, and 
•he color prindn. s«tem ! 7 ""^spond to RGB. Most likelj.. 

produce co'irrgl^Tn l^JhTcLt ^ '"T ^ '» 

provided a mapping be^.rcMS ;„ "'h'^ ""'^^^ "= 

even though the senLrt^^i;^ «lues. 
intensities ofI ^',I^"^/''r7''»« "=<>"'-« value by measuring the 
me tnree different encoding colors ROB within the hot spot 220. 

encoding colore cT^f,H '"^ *« "«e.ent 

on .heir o™ iS^^S^telS me^Ie ,he ho. spot 220, each taking 

^ vai^::! e^T^c i^^^ t^"irr"' r " 

provide die encoded content bL /.. "'^'^ "^en .ogeUier 

a hot spot w„^„k™r'^f!^lf'^ "f the Visual appears 

encoding coiors are m'Z^t Z^^:^'::^^^^ " H 

upon the content encodeH ti,«~- L I ^ ' depending 

levels and in sle^est^^^^^^^ '^^^ " 

instances the mtensity level of certain encoding colors would 
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^ -^e-r r~ ~ ~ - nr. spec., encod.. 
' -^dmm 30. Th. sensor 300 indudTa iLt °" ^^^^ ^ P^ysica! 
and a shroud 306. n.e lighremitter 3^'' ' ^^"^^^ elemenf 304 

LED2. and LED3. each oZlle e;™ "^'^ ^"""^"^ ^oTs 
y C3 respectively. TT,e sensing elemem IZ ^ ^l «>"-^ndmg to Cl. C2 
re^ponswe to the entire hght spect^ a Z Z^'^'T' ^""^^ ^^^^^ 
desired region 32 of the encoded physicalt^H fT" *^ ^00 with a 

on and bringing the Hght emitter ^02 "^'"^ ^ 

^sonably close proxinuty to tl^rdestd Tt \ '"^"^'^^ 304 into 

properly engaged with the desired t on 32 l'" "^^^ ^'^^ 30^ 
sensing element 304 from measm^nfex^l '^^^^ P«v<^nt the 

ambient light. ^"^"^ extraneous mformation in the form of 

108 for measu^, the 

put, the method 108 of Figure 6 seon^nf V "^"^ ''^ described. Simply 
the encoding color. Cl C2 and cTT' ^'""^'^ '"^^"^g intensitiro^ 
sensor 300 with the desi;ed^eSon ^ " ''''' ^^^^ '''' enZs th ' 

reflected intensity of Cl,l Te^ Ls^^Z'lT '"^^ ~ ^ 
measures the reflected intensity^f^ ^ ^24 turns LED2 on 

LED3 on. measures the i^fl^ted^r J^"^ ^^^^ off. a step 326 
Typically the sensing element 30?^^^/^ ^^ tums LED3 off 

^ the light mtensity and the s^o'^^Unrn 

^nverter. Thus the number of content L^df" ' ^^og'to-digital (A/D) 
ft^t encoding scheme is direcT T °" "^^'^ ^^^l^ble with the 
converter. "^^^^ ^^P^ndent upon the precision of the a!J) 

With reference to F* 
ffW'ding to y« another en.b'STJrflr'"'' 

™« second specw oncoding sT™! ° ° "HI b. described 
"-coded wiUun a ho. i^!t^.^'''^'"'"-'^>^^'''imor«^yT^ 

-«ne .0 .t::r^:sirrto^ur /d 
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«HT. H.. t ^ encoding scheme .o the encoding colors' intensities 

cqua, .0 a^ble^ *r~or; ''^^ " 
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S^St ^LlTMl"""'^™ ^^Sned a specific mLing or 

physical " °f 
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Turning next to Figure 10, a data linked book 350 in accordance with one 
embodiment of the present invention will now be described. A primary purpose 
of the linked book 350 is to link a physical book with data such as video and 
audio streams available via an information network such as the Internet. The 
linked data is then presented (e.g., displayed, played, etc.) on an Internet device 
such as a WebTV or a personal computer. 

The linked book 350 includes a physical book 352, a sensor 353 having 
an infrared transmitter 354, a plurality of pages such as page 356 and a plurality 
of hot spots such as hot spots 358. 360. and 362. The physical book 352 
appears conventional to a viewer in that the book 352 flips open to the different 
pages, each of which provide meaningful information in the form of text and 
graphics. In the example of Figure 10, the physical book 352 is opened to the 
page 356 entitled " Weather Report." Thus the user should immediately realize 
that the Weather Report page 356 is electronically linked to weather report 
information available over the corresponding information network. In the 
embodiment of Figure 10. each of the hot spots represents a uniform resource 
locator (URL). As will be appreciated, a URL is the addressing mechanism 
used by the Internet to correspond to a unique Internet address. A URL, 
together with any other desired information, is encoded within each hot spot 
according to a selected encoding scheme such as a spectral encoding or bar code 
scheme. 



When the user engages the sensor 353 with a desired hot spot, the sensor 
353 decodes the content of the hot spot, performs any necessary interpretation 
and other local functions, and then transmits the URL to the computer system 
25 370. The computer system 370 then uses the URL and other received 
information to download the desired data from the Internet, presenting such data 
to the user in the proper form. For example, a video stream may be displayed 
on the computer screen of the computer system 370. 

Turning next to Figure 1 1, a data linked globe 400 in accordance with yet 
30 another embodiment of the present invention will now be described. The data 
linked globe 400 includes both a sensor 402 having an infrared transmitter 404 
and a plurality of hot spots 406. The data linked globe 400 of Figure 1 1 
presents a spherical earth map. Encoded within the hot spots 406 are linking 
data. The linking data of Figure 1 1 may take any of a variety of suitable forms. 
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For example, similar to the data linked book of Figure 1 1, the linking data may 
include a URL. Each hot spot may represent a town, region, province, country, 
etc. The associated URL may direct the computer system 420 to an hitemet 
World Wide Web page produced, e.g-., by the Chamber of Commerce for thai 
town, region, etc. 

In an alternative embodiment, the computer system 420 of Figure 1 1 
maintains a database of geographical and/or historical data regarding the region 
represented by the hot spot. The linking data would then instruct the computer 
system 420 to present the corresponding information through the appropriate 
media interface, e.g., audio and video. In yet another embodiment, the linking 
data stored in each hot spot would contain the bulk of the content, the sensor 
402 simply transferring this content to the computer system 420 which would in 
turn present this information through the appropriate media interface. 

While this invention has been described in terms of several preferred 
embodiments and a number of specific examples, th^e are alterations, 
permutations, and equivalents which fall within the scope of this invention. 

For example, it is contemplated that in certain embodiments encoded 
content will be inserted onto a plurality of detachable bodies. A user provided 
with these may attach the bodies to a variety of different physical media. In this 
mariner, the user is, in essence, able to configure and program his own computer 
interface. As will be appreciated, the attachment mechanism of the detachable 
bodies may take any suitable form such as magnetic stripping, tape, hook-and- 
pile (e,g. Velcro®) members, etc. 

In another suitable embodiment, the marker 34 is a template that is 
positioned upon the encoded physical medivim 30. The template may be 
corifigurable such that the user can select the region 32. Alternatively, the 
template may affix to the encoded physical medium such that the region 32 is 
defined by the predetermined mating of the template and the encoded physical 
medium 30. 

As will be appreciated, the variety of physical medium upon which 
content may be encoded according to the present invention is almost limitless, 
ranging firom toys to tools to industrial parts and beyond. Still fiirther, the hot 
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It is also contemplated that the sensor 300 of Figure 5 may be suitably 
designed without a shroud 306. In some embodiments, the spectral encoding 
scheme tends to be insensitive to ambient light and thus the shroud 306 would 
be unnecessary. Additionally, the shroud 306 is removable by the user. In other 
embodiments, ambient light filter circuitry (or software) is included in the 
sensor 300 rendering the shroud 306 unnecessary. Still other embodiments of 
the sensor 300 include both the shroud 306 and filter circuitry. Those of skill in 
the art will well imderstand the design of shrouds and filter circuitry. 

Therefore it is desired that the appended claims be interpreted as 
including all such alterations, permutations, and equivalents as fall within the 
true spirit and scope of the present invention. 
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fVhat is claimed is: 



CLAIMS 

1. A method for interfacing a user and a computer system using an 
encoded physical medium, the encoded physical medium suitable for having at 
least one region wherein information has been encoded, the method comprising 
the steps of: 

a) measuring information present in a first region of the encoded 
physical medium; and 

b) determining whether the measured information present in the first 
region includes a marker indicating that certain information has been encoded in 
the first region. 



2. A method as recited in claim 1 wherein the step of measuring 
information present in a first region of the encoded physical medium includes 
the step of engaging a sensor with the first region of the encoded physical 
medium, the engaged sensor operable for measuring information present on the 
encoded physical medium. 



3. A method as recited in claim 2 wherein the sensor has an on and an 
off state and the step of engaging the sensor with the first region of the encoded 
physical medium includes the step of receiving a command requesting that the 
sensor be turned on. 



4. A method as recited in claim 2 wherein the marker is operable to 
generate light fi-om within a particular range of electromagnetic wavelengths 
and the sensor includes a sensing element responsive to the particular range of 
electromagnetic wavelengths, the step of engaging the sensor with the first 
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region of the encoded physical medium including positioning the receiver 
within the proximity of the first region. 



5. A niethod as recited in claim 4 wherein the sensor further includes 
5 a light emitter operable to emit light within the particular range of 
electromagnetic wavelengths. 



6. A method as recited in claim 5 wherein the operability of the 
marker to generate light from within the particular range includes the ability to 
10 reflect light from within the particular range. 



7, A method as recited in claim 4 wherein the operability of the 
marker to generate light from within the particular range includes the ability to 
emit light from within the particular range. 

15 

8. A method as recited in claim 4 wherein the step of determining 
whether the measured information present in the first region includes the marker 
includes the steps of: 

sensing light generated from the first region; 

20 determining whether the light generated from the first region includes 

light fix)m within the particular range, light from within the particular range 
indicating that the measured information present in the first region does include 
the marker and thus the certain encoded information. 



25 9. A method as recited in claim 8 wherein when it is determined that 

the first region does include the marker, the method further includes the step of 
decoding the certain information into certain decoded information suitable for 
use by the computer system. 
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10. A method as recited in claim 9 wherein the certain encoded 
information has been encoded according to a spectral encoding scheme, the 
receiver further responsive the spectral encoding scheme, and the step of 
engaging the sensor with the first region of the encoded physical medium 
includes positioning the sensor within the proximity of the first region. 



11. A method as recited in claim 10 wherein the spectral encoding 
scheme includes light outside the visible light spectrum. 

10 

12. A method as recited in claim II wherein the spectral encoding 
scheme includes infrared light. 



13. A method as recited in claim 10 further including the step of 
decoding the certain information encoded according to the spectral encoding 
scheme into certain decoded information suitable for use by the computer 
system. 



14 A method as recited in claim 9 wherein the certain encoded 
information includes informatioii encoded according to a bar code scheme, the 
receiver further responsive to the bar code scheme, and the step of engaging the 
sensor with the first region of the encoded physical medium includes 
positioning the sensor within the proximity of the first region. 



15. A method as recited in claim 1 wherein when it is determined that 
the measured information present in the first region includes certain encoded 
information, the method further includes the step of decoding the certain 
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encoded information into certain decoded information including a function to be 
performed by the computer system. 

16. A method as recited in claim 13 wherein the function to be 
performed by the computer system includes providing feedback to the user of 
the computer system. 



17. A method as recited in claim 16 wherein the feedback is generated 
immediately and locally by the computer system. 



18. A method as recited in claim 16 wherein the feedback provided to 
the user includes audio feedback. 



19. A method as recited in claim 16 wherein the feedback provided to 
the user includes visual feedback. 



20. A method as recited in claim 13 wherein the computer system is a 
local computer system and the function to be performed by the local computer 
system includes transmitting a control instruction to a remote computer system. 



21. A method as recited in claim 20 wherein the transmitted control 
instruction includes a uniform resource locator (URL). 



22. A method as recited in claim 21 wherein the computer system 
includes a display screen, the URL represents a particular web page available on 
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27. A method as recited in claim 25 wherein satisfaction of the specific 
condition depends upon the stored information, the method further including the 
steps of: 

g) determining that the stored information is such that the specific 
5 condition is satisfied; and 

h) in response to the satisfaction of the specific condition, performing 
an operation that is a ftmction of at least some of the stored data. 



28. A method as recited in claim I wherein the encoded physical 
10 medium includes at least one page having text and graphics. 

29. A method as recited in claim 1 wherein the encoded physical 
medium includes a spherical object. 

15 30. A method as recited in claim 1 wherein the encoded physical 

medium includes an article of apparel. 

31. A method as recited in claim 1 wherein the encoded physical 
medium includes packaging material. 

20 

32. A method as recited in claim 1 wherein the encoded physical 
mediimi is selected by the user ft-om local surroundings. 

33. A method as recited in claim 1 wherein the at least one region 
25 provides a visual indication to the user that information has been encoded 

therein. 
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39. A method for generating an encoded physical medium as recited in 
claim 36, the step of encoding the content together with the marker including 
the steps of: 

generating a binary number that represents the content; and 

5 encoding the binary number that represents the content according to a 

spectral encoding scheme. 



40. A method for generating an encoded physical mediiun as recited in 
claim 39 wherein the marker represents ink that reflects light from within a 
1 0 particular range of electromagnetic wavelengths. 



41. A method for generating an encoded physical medium as recited in 
claim 36 further including the step of designing text and graphics within the 
representation of the encoded physical medium. 

15 

42. A method for generating an encoded physical medium having a 
region wherein content has been encoded, the method comprising the steps of: 

receiving content that is to be encoded into a desired location on the 
encoded physical medium; 

20 encoding the content according to a particular encoding scheme suitable 

for application onto the encoded physical medium; and 

inserting the encoded content and a marker into the desired location of the 
encoded physical medium, the marker indicating that the content is encoded 
within the desired location. 

25 

43. A computer interface for interfacing a user and a computer system 
using an encoded physical medium, the encoded physical medium suitable for 
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15 



20 



25 



having at least one region wherein information has been encoded, the computer 
interface comprising: 

a sensor operable for measuring information present on the encoded 
physical medium; and 

a first device coupled to the sensor and responsive to determine whether 
information measured by the sensor includes a marker indicating that certain 
encoded infomiation is present in the measured information. 



44. A computer interface as recited in claim 43 further comprising a 
second device responsive to the first device such that when the first device 
determines the presence of the marker, the second device is operable to decode 
the certain encoded information present in the measured information. 



45. A computer interface as recited in claim 44 further comprising a 
transmitter device operable to transmit the certain decoded information to the 
computer system. 



46. A computer interface as recited in claim 44 further comprising: 

a third device operable to process the certain decoded information and 
generate formatted data suitable for transmission to the computer system; and 

a transmitter device operable to transmit the formatted data to the 
computer system. 



47. A computer interface as recited in claim 43 further comprising a 
memory device suitable for storing the certain encoded information. 
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54. An encoded physical medium suitable for use in interfacing a user 
and a computer system, the encoded physical medium including a region 
wherein certain information is encoded, the certain encoded information 
mcluding a marker indicating that the certain encoded information is encoded in 
the first region, the certain encoded information including data suitable for 
interpreting into computer readable data. 



55. An encoded physical medium as recited in claim 54 wherein the 
marker is operable to generate light from within a particular range of 
10 electromagnetic wavelengths. 



56. An encoded physical medium as recited in claim 55 wherein the 
marker is operable to generate infrared light. 



15 



57. An encoded physical medium as recited in claim 55 wherein the 
data suitable for decoding into computer readable data is encoded according to a 
spectral encoding scheme. 



58. An encoded physical medium as recited in claim 55 wherein the 
20 data suitable for decoding into computer readable data is encoded according to 
bar code scheme. 



59. An encoded physical medium as recited in claim 54 .wherein the 
data suitable for decoding into computer readable data inchides tcxLpresented 
25 upon the encoded physical medium and the marker is ink applied over the 
selected text. 
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72. An electronic data linked physical medium as recited in claim 70 
wherein the linking data is encoded according to a spectral encoding scheme 
and the at least one hot spot further includes a marker indicating that the linking 
data is encoded within the hot spot, the sensor further operable to detect the 
presence of the marker. 

73. An electronic data linked physical medium as recited in claim 70 
wherein the remote computer system is a personal computer coupled to the 
Internet, and the Unking data includes a uniform resource locator (URL). 



74. An electronic data linked physical medium as recited in claim 70 
wherein the remote computer is a WebTV coupled to the Internet, and the 
linking data includes a uniform resource locator (URL). 

75. An electronic data linked physical medium as recited in claim 70 
wherein the hot spots are removably attached to the electronic data linked 
physical medium. 

76. A system comprising a plurality of electronic data linked physical 
mediums as recited in claim 75, the hot spots suitable for removably attaching 
to each of the electronic data linked physical mediums, wherein the user is 
capable of creating a variety of different configurations for the system. 

77. An electronic data linked physical medium as recited in claim 70 
wherein the physical medium is a globe-shaped map. 

78. An electronic data linked physical medium as recited in claim 77 
wherein the remote data retrieved via the at least one hot spot is historical data 
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corresponding to the region of the globe-shaped map over which the at least one 
hot spot lies. 



79. An electronic data linked physical medium as recited in claim 77 
5 wherein the remote data retrieved via the at least one hot spot is geographical 
data corresponding to the region of the globe-shaped map over which the at 
least one hot spot lies. 



80. An electronic data linked physical medium as recited in claim 77 
10 wherein the remote data retrieved via the at least one hot spot is a URL 
corresponding to a web page for the region of the globe-shaped map over which 
the at least one hot spot lies. 



81, An electronic data linked, physical medium as recited in claim 70 
15 wherein the physical medium is an item of apparel. 



82. An electronic data linked physical medium as recited in claim 81 
wherein the at least one hot spot is disposed on the item of apparel such that the 
at least one hot spot coincides with a predefined portion of a wearers anatomy, 
20 the remote data retrieved via the at least one hot spot being anatomical data 
related to the predefined portion of anatomy. 



83. An electronic data linked physical medium as recited, in claim 70 
wherein the at least one hot spot takes the form of a tattoo on the user. 

25 

84. An electronic data linked physical medium as recited in claim 83 
wherein the content encoded in the at least one hot spot is related to the user. 
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85. An electronic data linked physical medium as recited in claim 84 
wherein the content encoded in the at least one hot spot is a URL of a home web 
page of the user. 
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100 



102' 



PROVIDE USER WITH AN ENCODED 
PHYSICAL MEDIUM AND A SENSOR SUITABLE 
FOR SENSING INFORMATION PRESENT ON 
THE ENCODED PHYSICAL MEDIUM 



112 



>■<- 



104' 



USER EXPLORES THE ENCODED PHYSICAL 
MEDIUM AND SELECTS A DESIRED REGION 
OF THE ENCODED PHYSICAL MEDIUM 



106' 



USER ENGAGES THE SENSOR WITH 
THE DESIRED REGION OF THE 
ENCODED PHYSICAL MEDIUM 



108' 



SENSOR MEASURES INFORMATION PRESENT 
^ THE DESIRED REGION OF THE ENCODED 
PHYSICAL MEDIUM 
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PERFORM ANY ACTION 
CORRESPONDING TO 
A NULL MEANING 



YES 



DOES THE MEASURED INFORMATION 
HAVE NULL MEANING? ^ 



NO 
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SENSOR INTERPRETS 
MEASURED INFORMATION 



116' 



SENSOR PERFORMS AC TION INDICATE D BY 
COMBINATION OFINTERFFtETED 
INFORMATION AND CONTEXT 



Figure 2 



SUBSTITUTE SHEET (RULE 26) 
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102 



200' 



DESIGNER CREATES REPRESENTATION OF 
PHYSICAL MEDIUM THAT WILL INCLUDE "HOT 
SPOTS" AND MAY INCLUDE TEXT AND/OR 
GRAPHICS 



202' 



DEFINE CONTENT OF HOT SPOTS 



204 



ENCODE CONTENT OF HOT SPOTS 



206' 



INSERT ENCODED CONTENT INTO THE 
APPROPRIATE LOCATION(S) WITHIN THE 
REPRESENTATION OF THE PHYSICAL MEDIUM 



208' 



GENERATE AN ENCODED 
PHYSICAL MEDIUM FROM 
ITS REPRESENTATION 



( ^ONE ^ 



Figure 3 
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322' 



324' 



326' 



Figure-4 



300- 




-30 



Figure 5 



320^^ ENGAGE SENSOR WITH HOT SPOT 



TURN LED1 ON, ME 
OFCl.TUR 


LASURE INTENSITY 
N LED1 OFF 


> 


f 


TURN LED2 ON. MEASURE INTENSITY 
OF 02. TURN LED2 OFF 


> 




TURN LED3 ON. MEASURE INTENSITY 
OF 03. TURN LED3 OFF 




DONE 

Figure 6 
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Figure 7 




Figure 8 
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204 



310' 



GENERATE BINARY NUMBER THAT 
REPRESENTS THE CONTENT OF THE 
PRESENT HOT SPOT 



312' 



ENCODE BINARY NUMBER THAT 
RFPRESENTS THE CONTENT OF THE 
PRESENT HOT SPOT ACCORDING 
TO THE SPECTRA! ENCODING SCHEME 



(^D^E^ 



Figure 9 
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Figure 11 



